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Description 




Method and circuit arrangement for picture-in-picture 
insertion 

5 

The invention relates to a method for picture-in- 
picture insertion, in which a sequence of decimated 
inset pictures is written to a memory with at least two 
segments and is read out for insertion into a sequence 
10 of main pictures, to be precise in accordance with the 
preamble of claim 1, and also a circuit arrangement for 
picture-in-picture insertion, in particular for 
carrying out said method in accordance with the 
preamble of claim 6 . 



Various methods and, apparatuses for inserting one or 
more inset pictures into a main picture (PIP - Picture 
In Picture) are known. In this case, the inset pictures 
stored in the memory are read out synchronously with a 
fU 2 0 main picture. Since the read-out speed is generally 



higher, in a manner corresponding to the decimation of 
the inset picture, than the write-in speed, a seam can 
M occur in the inset picture on account of the write 

pointer being overtaken by the read pointer/ since the 

2 5 inset picture is then composed of a current part and a 

preceding part . 

In particular in the case where the two parts originate 
from different motion phases, a disturbing effect is 

3 0 produced since moving objects through which the seam 

passes are displayed in a distorted manner. Moreover, 
if the frequencies of the inset and main pictures do 
not correspond exactly, the seam drifts, which is 
perceived as particularly unpleasant. 



EP 0 739 130 A2 discloses, for the purpose of avoiding 
this problem, storing two inset pictures (or fields in 
each case) in the memory device, so that it is always 
the case that exactly one picture can be read out while 
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the next picture is written. Although this prevents the 
write pointer from being overtaken by the read pointer, 
there is nonetheless a significant disadvantage in that 
the storage capacity must be very high, which is 
5 associated with considerable costs. 

The invention is based on the object, therefore, of 
providing a method and also a circuit arrangement for 
picture-in-picture insertion of the type mentioned in 
10 the introduction by means of which, with relatively low 
outlay, an inset picture can be generated without a 
seam. 

This object is achieved by means of a method which is 
15 distinguished by the fact that: 

the inset pictures are written to the memory 
device in a circulating manner under continuously 
incremented write addresses, 

the first address of each written-in inset picture 
20 is stored, 

an overtake signal is formed by comparing the 
instantaneous address with a previously stored address, 
said signal indicating whether a previous address has 
been reached again and, consequently, the corresponding 
25 picture content has been overwritten, 

by evaluation of the overtake signal, the current 
or preceding segment is selected for read-out depending 
on whether or not overtaking took place before the 
start of the read-out, and 
30 - the inset picture stored in the selected segment 
is read out with continuously incremented read 
addresses and is inserted into the main picture. 

The object is furthermore achieved by means of a 
3 5 circuit arrangement for inserting a sequence of 
decimated inset pictures into a sequence of main 
pictures, which has a memory device having at least two 
segments for the inset pictures, a write controller and 
a read controller and which is distinguished in 
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particular by the fact that an overtake signal that can 
be fed into a display controller can be generated by 
the write controller, in that a segment buffer is 
provided, which segment buffer is connected to the read 
5 controller and serves to store a first and a last 
address of an inset picture, and in that a memory 
segment to be read out by the read controller can be 
selected by means of the display controller in a manner 
dependent on the overtake signal. 

10 

The solutions according to the invention are based on 
the insight that the write pointer can be prevented 
from being overtaken by the read pointer through 
suitable circulating addressing of the memory device in 

15 a manner utilizing the speed difference between the 
pointers. A significant advantage of this solution is 
that this holds true even when the size of the inset 
picture changes. Furthermore, is also not necessary to 
provide completely separate memory areas for reading 

20 and writing or to adapt the organization of the memory 
to the inset-picture size. 

The subclaims contain advantageous developments of the 
invention . 

25 

Accordingly, for example, the write and read addresses 
are continuously incremented from a first memory 
address up to a last memory address and are in each 
case reset to the first memory address again after the 
30 last memory address has been reached. 

For this purpose, in the circuit arrangement according 
to the invention, it is preferably provided that the 
write controller and the read controller each have an 
3 5 address counter for incrementing the write addresses 
and read addresses, respectively. 

Further details, features and advantages of the 
invention emerge from the following description of 
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preferred embodiments with reference to the drawings, 
in which: 

Figure 1 shows a block diagram of a circuit 
arrangement according to the invention; 

Figure 2 shows a diagram of the signal profiles when 
writing to a picture memory; 

Figure 3 shows a diagram of the signal profiles when 
reading from a picture memory; 

Figure 4 shows a representation for illustrating the 
origination of a seam; 

Figure 5 shows a diagram of' the signal profiles 

according to the inventions- 
Figure 6 shows a block diagram of a circuit 

arrangement for selecting a memory segments- 
Figure 7 shows an address pointer representation in 

the absence of vertical decimations- 
Figure 8 shows a representation of memory division in 

the absence of vertical decimation, and 

Figure 9 shows a representation of the memory outlay 
in the case of decimation. 

In accordance with Figure 1, a main picture signal is 
fed to a circuit arrangement according to the invention 
via a first input A, and an inset picture signal is fed 
via a second input B. The inset picture signal passes 
to a decimation device 1 and also to a synchronization 
device 8. The picture signal of the inset picture, 
which is generally vertically decimated, is buffer- 
stored in the form of fields in a memory 2 and, after 
read-out, is fed together with the main picture signal 
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to an insertion apparatus 3, which generates a 
composite picture signal. 

The synchronization device 8 is connected to the 
5 decimation device 1 via a first output and to a write 
controller 4 via a second output. A first output of the 
write controller 4 is connected to the memory 2, while 
a second output is connected to a segment buffer 7 and 
a third output is connected to a display controller 6. 

10 The output of the segment buffer 7 is connected to a 
read controller 5, to which a first output of the 
display controller 6 is also fed. An output of the read 
controller 5 is connected to the memory 2 . Finally, a 
second output of the display controller is fed to the 

15 insertion apparatus 3, the display controller having an 
input C for synchronization signals of the main 
picture. 

In order to write to the memory 2 an inset picture 
20 signal that has been decimated by the decimation device 
1, the write controller 4 generates the required 
addresses. In the simplest case, given an e.g. linear 
organization of the memory 2, the write address pointer 
is incremented after each write operation. If the 
25 address pointer reaches the last address, it is reset 
to the first address, under which the writing process 
is continued, this being done over the duration of the 
field with interruptions in accordance with the 
decimation. In this case, a preceding picture (field) 
30 is inevitably overwritten. 

With each beginning of a new field, the first write 
address is stored in the segment buffer 7, which 
additionally stores the last address as well. 
35 Furthermore, the field position and size are also 
stored in the form of lines and pixels per line in the 
segment buffer 7, the data respectively stored last not 
being overwritten. The storage capacity of the segment 
buffer 7 is thus about double said data to be stored 
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for a field, which in each case corresponds to a field 
in an instantaneous segment and a current segment in 
the memory 2 . 

Each time the write pointer is incremented, the write 
controller 4 additionally checks whether the new 
address has a specific offset, with respect to the 
previously stored (not with respect to the currently 
stored) address. This offset may also be zero in the 
case of a high vertical decimation factor. It serves 
essentially as a safety margin in order to avoid 
overtaking as a result of asynchronous data acceptance 
or slightly deviating vertical frequencies. With this 
offset, an overtake signal is generated which indicates 
whether the address has been reached again and hence 
this picture content has been overwritten. 

The content of the segment buffer 7 is transferred to 
the read controller 5, by means of which the memory 2 
is read beginning at one of the two addresses present 
after the start of reading at the insertion position, 
so that the inset picture can be generated in 
accordance with its position and size in the main 
picture by the insertion apparatus 3 . 

On the basis of the overtake signal transmitted by the 
write controller 4, the display controller 6 decides 
which of the two segments present in the memory 2 is 
read out. The selection of the segment initially always 
proceeds from the last segment. If overtaking took 
place before the start of reading, then the 
instantaneous (current) segment is read out. A 
corresponding segment selection signal is transmitted 
to the read controller 5. 

The insertion position is also calculated by means of 
the display controller. The display controller 6 
furthermore serves for correcting the raster positions, 
this being effected by comparison between the raster 
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position of a displayed picture and the raster position 
of a stored picture and skipping or repetition of a 
line. Finally, the display controller 6 can also be 
utilized for multi-picture insertion. 

Before the algorithm is described in detail, firstly 
the terms used in this context will be explained. 

As was explained above, the decimated pixels of the 
inset picture are firstly stored in the memory 2 . For 
display purposes, they are read from the memory again 
in the temporal framework of the main picture. For this 
purpose, the write and read addresses have to be 
generated by separate address counters. 

Figure 2 shows the corresponding signal profiles when 
writing to the memory 2 . The instantaneous value of the 
write address counter represents a write pointer with a 
decimated inset pixel in the memory 2. The terms "write 
pointer" and "read pointer" shall be used for this 
reason . 

An acquisition window (vaqwin) begins with a pulse 
(vaqstart) , and the write address pointer rises with 
each decimated pixel. Since only the visible part of 
the inset picture is decimated, the write address 
pointer rests whenever there is no valid line available 
from the vertical decimation stage. After the last line 
to be decimated, the pointer is reset to the start 
position. The picture of a ramp with shoulders results 
in the representation. 

Figure 3 shows the corresponding signal profiles, when 
reading from the memory 2. The instant at which a start 
signal (vdisstart) must be generated in order to begin 
reading from the memory 2 is calculated by means of the 
display controller 6 on the basis of the desired 
insertion position of the decimated inset picture in 
the main picture. With each line of the main picture, 
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at the corresponding insertion position (hdisstart) , a 
line of the decimated inset picture is then read from 
the memory and inserted into the main picture (video 
mux) . 

5 

The picture of a ramp with shoulders once again results 
in the representation of the read address pointer. The 
ramps of the read and write address pointers differ in 
respect of their average gradient. The latter will be 
10 considered in detail below since it represents the rate 
at which a number of memory cells are swept over within 
a specific .period of time. 

The speed of the write address pointer changes with the 
15 decimation factor, to be precise in such a way that the 
ramp of the write address pointer becomes less steep as 
the decimation factor rises. 

Figure 4 shows the origination of a seam for the case 
20 of representation in the frame mode. If only a field 
memory is present and the inset picture has been 
decimated, the write pointer is generally overtaken by 
the read pointer. Since the sources for the inset 
picture ( "insert_f ield" ) and the main picture 
25 ( "parent_f ield" ) are generally asynchronous with 
respect to one another, a seam is thus produced. In 
Figure 4, this is indicated by the point of 
intersection between the relatively slow write pointer 
and the read pointer that overtakes the latter, the 
3 0 upper-case letters A, B designating the raster position 
of the inset picture and the lower-case Greek letters 
lSi/ S6 designating the raster position of the main 
picture . 

3 5 From a temporal standpoint, the current field is read 
before the seam, while part of an older picture is 
reproduced after the seam. It should be noted in this 
case that, as a result of the line interlacing method 
(interlaced mode) , the raster position changes in the 
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event of overtaking, and this subsequently has to be 
corrected again. 

The seam becomes very clearly visible in particular 
when a moving picture is displayed, that is to say when 
the current picture and the older picture contain 
different motion phases. If different standards are 
used for the main picture and the inset picture, then 
rolling of the seam occurs . As a result of different 
frame frequencies, the combination of the raster 
positions at the beginning of the representation 
changes a number of times every second. It is a very 
complicated procedure to perform a correction in a 
manner dependent on this incorrect position. Moreover, 
the picture can be very erratic in the vertical 
direction and be perceived as unpleasant. 

The method according to the invention and the circuit 
arrangement according to the invention now allow, in 
particular in the frame mode, a seamless representation 
of pictures that have been decimated to different 
extents, without two separate field memories having to 
be available. 

For the explanations below, it will be assumed that 
orthogonal memory division has been chosen. As a 
result, the memory is divided into lines with fixed 
start addresses. The length of such a line in the 
memory is determined by the largest picture. 

Furthermore, it will be assumed that the inset picture 
and the main picture are present in the same standard. 
The two picture sources can be asynchronous but, in 
terms of their time frame, should initially have no 
deviations from one another. If the inset picture is 
not decimated vertically and horizontally, one field 
memory suffices for generating a seamless picture-in- 
picture representation, since the two address pointers 
cannot overtake one another on account of their 
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identical speed. The case of raster position correction 
forms an exception. This correction is performed in the 
first line given a corresponding combination of main 
and inset picture positions. 

In this case , the read address pointer j umps by one 
line. In the course of this jump, the situation where 
the write address pointer is overtaken must be 
prevented. This is achieved in that there is space for 
two additional lines in the memory. 

Furthermore, suitable control of the memory accesses is 
necessary. The memory is written to with the fields in 
a circulating manner. As a result, with each new field, 
the start address is shifted by the number of 
additionally present lines in the direction of lower 
physical addresses. If the physical end address of the 
memory is reached in the course of writing, then a jump 
is made back to the start address. 

Figure 5 shows the corresponding signal profiles during 
write control (a) and read control (b) . Accordingly, 
there is thus always somewhat more than one inset field 
in the memory. In a similar manner to the case of the 
organization of two field memories, in this case, too, 
the write address pointer determines the memory segment 
enable. The signals "vaqstart", NN vaqwin", "vdisstart" 
and "hdisstart " again have the same meaning as in 
Figures 2 and 3 . 

Figure 6 shows a block diagram for selecting the 
respectively valid memory segment. The circuit 
comprises an address counter 12, whose output is 
connected to a first register 10 and to a first input 
of a comparator 14, a second register 11, to whose 
input the output of the first register 10 is connected, 
and also a multiplexer 13, whose first input 
(line_adr_next ) is connected to the output of the first 
register 10 and whose second output is connected to the 
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output of the second register 11. This latter output is 
also connected to a second input of the comparator 14, 
whose output is connected to a flip-flop 15. 

For control of the read accesses, the start addresses 
of the old and new inset fields are stored in the 
registers 10, 11. With the beginning of a new field, 
the older of the two register contents is rejected, and 
the formerly new start address becomes the old address, 
while the present current address becomes the new start 
address. If the beginning of the older inset field has 
been overwritten as a result of the memory being 
written to in a circulating manner, it can no longer be 
read. 

For selection of the valid memory segment, the content 
of the address counter 12 ( line_address) is continually 
compared with the start address of the older of the two 
fields (line_address_cur) in the comparator 14. In the 
event of correspondence, the flip-flop 15 is set and 
the start address of the new field is then present at 
the output of the multiplexer 13 . If a new field is 
begun, then the flip-flop is reset, and, as a result of 
the change of the register contents, the same start 
address as before is present at the output of the 
multiplexer 13, until this is also overwritten again. 
In this way, the memory enable points to a valid memory 
segment at every instant. The memory space that is 
additionally present means that the read pointer cannot 
reach or overtake the write pointer even in the event 
of a jump on account of the raster position correction. 

In order to satisfy the general requirements, however, 
this sequence must be extended. Considerable deviations 
from the standard can occur particularly in the case of 
video recorders which are operated with fast forward or 
rewind with picture reproduction. In this case, by way 
of example, it is also necessary to take account of the 
maintenance state and the wear of the tape material. 
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The memory control must be able to compensate for the 
effects of a stretched tape and also synchronism 
fluctuations of the drive mechanism. However, a 
precondition in this case is that the sync pulse 
separation still operates correctly in the case of such 
a signal. 



The vertical frequency f v and the horizontal frequency 
f H are related through the number Z of lines as 
follows : 



fv = f H / Z (5.1) 

The line frequencies of the main picture (f Hp ) and of 
the inset picture (ffu) are of interest for the 
algorithm. Their fluctuations directly affect the 
writing and reading speed. The larger the line 
frequency f Hi of the inset picture, the more memory 
content is written per unit time. The smaller the line 
frequency f Hp of the main picture, the fewer lines are 
read per unit time. The opposite correspondingly holds 
true . 



If the same standard is used in both sources, then the 
following relationships hold true, where f H is the 
desired line frequency: 



fHi_max = f H (1 + df Hi ) (5.2) 

fHi.min = f H (1 + df Hi ) (5.3) 

fHp_max = f H d + dfHp) (5.4) 

fHp.min = f H (1 + df Hp ) (5.5) 



It will additionally be assumed that no vertical 
picture decimation is performed. Since the sources are 
again two sources that are asynchronous with respect to 
one another, write and read pointers can adopt any 
desired position with respect to one another. Equally, 
the pointers can overtake one another in both 
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directions given corresponding combination of the 
horizontal frequencies . 

Figure 7 shows the memory lines required for the 
5 respective write and read pointers, where Z aC q is the 
number of picture lines of a field that are used for 
acquisition. The representation makes it clear how many 
lines must additionally be present in the memory in 
order to prevent overtaking of the pointers in both 
10 directions. 

Z S 2 - Z acq { ( f Hi_max ~ f Hp_min ) / f Hi_max } (5.6) 
Z s l = Z aC q ^ ( f Hp_max ~ f Hi_min ) / f Hp_max } (5.7) 

15 The total amount of additionally required memory for 
the seamless picture-in-picture representation turns 
out to be: 

Z g = Z acq + Z s i + Z S 2 (5.8) 

20 

During writing, the memory contains part of an old 
field and part of a new field. Through the position of 
the write address pointer, one of the two fields is 
enabled for reading. If fewer than Z sl lines of the new 
25 field have been written, the old field is enabled. 
Otherwise, the distance from the beginning of the old 
field is less than Z s2 lines, with the result that the 
new field can be read. 

30 Figure 8 shows the memory division in the case where no 
vertical decimation is performed. 

If decimation is then effected in the horizontal and 
vertical direction, this influences the speed of the 
35 write address pointer. The corresponding relationships 
are shown in Figure 9. The rise of the ramp is smaller 
in this figure. 
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It will . initially be assumed that the writing speed 
changes to a considerable extent as a result of the 
decimation. By contrast, the fluctuations due to 
changes in the line frequency shall be small. 

5 

It follows from this that the read pointer can no 
longer be overtaken by the write pointer on account of 
the speed difference. As a result, the memory outlay 
for seamless picture-in-picture representation can now 
10 be determined using the decimation factors and the 
fluctuation range of the picture sources. 

^sl dec 

(deC ver ) = (Zacq/deCver) d~fHi mindeC ver / f Hp max) 

(5.9) 

15 

The additional requirement of lines decreases as the 
decimation factor increases. The maximum emerges for a 
vertical decimation factor dec ver of 1 (vertically 
undecimated picture) . 

20 

The possibility of the read pointer being overtaken by 
the write pointer will now also be taken into account. 
For small vertical decimation factors, the result is a 
further additional memory requirement of: 

25 

^s2 dec — V ^acq /deC V er) (1 -LHp min dec ver /f H i max ) (5.10) 

For a . meaningful result, the expression in the right- 
hand brackets must be positive. The validity of this 
30 expression is thus limited to a range of: 

"1" less than equal to M dec ver " less than equal to 

( f Hi_max/ f Hp_min) " (5.11) 

35 For values which are greater than the right-hand limit, 
Zs2_dec shall be set to zero. 

Taking account of the validity ranges, the additional 
memory requirement results from the sum of Z s i d e C and 
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Z S 2_dec- The examinations made at the beginning are a 
special case for a vertical decimation factor of 1. 

The total memory requirement consequently turns out to 
be : 

Zg(dec V er) = Zacq/decver + Z s i_dec + 2 s2 _dec for "1" less than 
equal to "dec ve r" less than equal to " f H i_max/f H P _min) " 

(5.12) 

thus resulting in the following: 

Zg(deCver) = Zacq/deCver ( 3 — fHp_min deC V er/f Hi_max *~ 
fHi_min/ (deCver fHp_max) (5.14) 

Otherwise, the following holds true: 
Zg(deCver) = Z acq /dec V er + Z s i_d e c for 

"dec V er" greater than (fHi_max/ fHp_m±n) (5.13) 
The following results from this: 

Zg(deCver) = Zacq/deC V er (2 - f H i_min/ (deC ve r f Hp_max ) 

(5.15) 

The total number of memory cells required has its 
maximum for a vertical decimation factor of 1. As the 
decimation factor increases, the memory cell 
requirement greatly decreases. 

It shall also supplementarily be pointed out that when 
the memory 2 is extended to three segments, the method 
according to the invention can also be employed with a 
frame frequency of 100 Hz in the AABB raster. 



